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ABSTRACT 

Objective: To prevent wrong surgery, the WHO 'Safe 
Surgery Checklist' was introduced in 2008. The 
checklist comprises a time-out procedure (TOP): the 
final step before the start of the surgical procedure 
where the patient, surgical procedure and side/site are 
reviewed by the surgical team. The aim of this study is 
to evaluate the extent to which hospitals carry out the 
TOP before anaesthesia in the operating room, whether 
compliance has changed over time, and to determine 
factors that are associated with compliance. 
Design: Evaluation study involving observations. 
Setting: Operating rooms of 2 academic, 4 teaching 
and 12 general Dutch hospitals. 
Participants: A random selection was made from all 
adult patients scheduled for elective surgery on the day 
of the observation, preferably involving different 
surgeons and different procedures. 
Results: Mean compliance with the TOP was 71.3%. 
Large differences between hospitals were observed. No 
linear trend was found in compliance during the study 
period. Compliance at general and teaching hospitals 
was higher than at academic hospitals. Compliance 
decreased with the age of the patient, general surgery 
showed lower compliance in comparison with other 
specialties and compliance was higher when the team 
was focused on the TOP. 

Conclusions: Large differences in compliance with 
the TOP were observed between participating hospitals 
which can be attributed at least in part to the type of 
hospital, surgical specialty and patient characteristics. 
Hospitals do not comply consistently with national 
guidelines to prevent wrong surgery and further 
implementation as well as further research into non- 
compliance is needed. 



INTRODUCTION 

Ideally, hospitals should be safe environ- 
ments for their patients. However, making 
errors is inherent in all humans.^ The report 
'To Err is Human' showed that errors cause 
44 000-98 000 deaths and over one million 



Strengths and limitations of this study 



Structured observations of compliance with 
time-out procedure (TOP) and factors that are 
associated with compliance at operating rooms 
(ORs). 

The presence of the observer might have influ- 
enced the behaviour of the OR staff. 
A potential selection bias in the surgical proce- 
dures on the observation days may have 
occurred. No outcome data are available. 



injuries each year in American hospitals.^ As 
a result, patient safety became a major topic 
on the healthcare agenda.^"^ Patient safety 
covers the prevention of errors and adverse 
events associated with healthcare that affect 
patients.'^ An adverse event is unintentional 
harm caused by healthcare management 
rather than by the patient's underlying 
disease that results in a prolonged hospital 
stay, temporary or permanent disability or 
death.'' In 2004, adverse events occurred in 
approximately 5.7% of hospital admissions in 
the Netherlands: approximately 2.3% of the 
adverse events were potentially preventable.'' 
More than 54% of the unintentional adverse 
events were associated with the surgical pro- 
cedure, of which 34% were reviewed as being 
preventable.*' It is therefore important to 
ensure and improve patient satfety during 
surgery. 

Patient Sctfety in surgery has several aspects. 
One of these aspects is wrong surgery, which 
can be classified into three groups: surgery at 
the wrong site, surgery on the wrong patient 
and carrying out the wrong procedure.^ 
Wrong site surgery occurs whenever a 
planned surgical procedure is performed at 
or on the wrong place, part and side or site. 
Wrong patient surgery refers to a procedure 
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performed on the wrong patient. Wrong procedure 
surgery refers to a different procedure being performed 
than the one planned for the patient. The Joint 
Commission on Accreditation of Healthcare 
Organizations (JCAHO) sentinel event database ranked 
wrong site surgery as the second most frequendy 
reported adverse event between 1995 and 2005.^ In the 
USA, for instance, the estimated rate of wrong site 
surgery ranges from 0.09 to 4.5/10 000 operations.'^ 

To prevent wrong surgery, the JCAHO guideline 
'Universal protocol for Preventing Wrong Site, Wrong 
Procedure, Wrong Person Surgery' was adopted in 2003 
by the Joint Commission in the USA.^^ Consequendy, 
the WHO introduced a checklist in 2008 for worldwide 
use, called the 'Safe Surgery Checklist'. In 2009, the 
WHO concluded that the use of a checklist in the oper- 
ating room (OR) is associated with a significant decrease 
in postoperative complication (30%) and mortality rates 
(50%). Based on these results, the WHO estimated 
that implementing the checklist could save 500 000 lives 
every year worldwide.^ ' Other studies provided evidence 
supporting the use of surgical checklists as well.'*^^^ In 
the Netherlands, the SURgical PAtient Safety System 
(SURPASS) was developed with the same intention. It is 
based on safety checks used in the aviation industry to 
reduce human error.^" Research on the external valid- 
ation of the SURPASS shows a reduction in uninten- 
tional harm.^^"^'^ 

Each of the checklists atforementioned comprises a 
time-out procedure (TOP). Errors can be avoided by 
including a preoperative discussion just before the start 
of the surgical procedure. This takes place during a 
time-out involving a review of the names and roles of all 
team members, characteristics of the patient, the oper- 
ation plan, familiarity with the procedure, the presence 
of the correct materials/equipment and potential issues 
for the patient.^* Although evidence is scarce, it is 
likely that these TOPs reduce uncertainties in the OR 
among the surgical team and reduce the risk of wrong 
surgery. The TOP is the final step before the start of the 
surgical procedure and is therefore crucial in preventing 
wrong surgery. ATOP is carried out just before anaesthe- 
sia,^^ and consists of three checks (the patient, the pro- 
cedure and the side/site), all of equal importance in 
preventing wrong surgery. 

The aim of this study is to evaluate the extent to 
which hospitals carry out the TOP before anaesthesia in 
the OR, whether compliance has changed over time, 
and to determine factors that are associated with the 
TOP compliance. Insights into compliance with the 
TOP and the factors associated with compliance are 
important because they have the potential to improve 
the TOP and reduce adverse events in surgical processes 
throughout the world. This study was carried out in the 
Netherlands and was part of a larger evaluation study of 
the Dutch Hospital Patient Satfety Program (hereinafter 
'Safety Program') that was carried out during the final 
year of the programme (box 1). 



Box 1 The Dutch Hospital Patient Safety Program. 



The Dutch Hospital Patient Safety Program (Safety Program) was 
set up in 2008 to reduce preventable unintentional adverse events 
in Dutch hospitals by 50% by the end of 201 2.^'' The Safety 
Program consisted of 10 patient safety themes and clinical guide- 
lines were developed for each theme. Hospitals were given 
5 years to implement these guidelines. One of the themes was 
prevention of wrong surgery. There are several risk factors for 
wrong surgery, for example, insufficient compliance, inadequate 
identification and verification and bad preoperative planning.^' 
The Safety Program therefore instructed the participating Dutch 
hospitals to implement several steps to decrease wrong surgery, 
based on the SURPASS checklist. One of the steps is identifica- 
tion and verification by means of a TOP consisting of checks on 
the correct patient, correct side, and correct intervention.^^ 



On the basis of the goals of the Safety Program, it was 
expected that the compliance with the TOP would 
increase over time and would become more visible 
during the final year of the programme when hospitals 
approached the public deadline at the end of 2012. 

The research questions are: 

1. To what extent do Dutch hospitals comply with the 
TOP before anaesthesia in the OR? 

2. How has the compliance with the TOP changed 
during the final year of the Safety Program? 

3. What factors are associated with compliance with the 
TOP? 

METHODS 
Study design 

This study was part of a larger evaluation study of the 
Safety Program that was carried out between November 
2011 and December 2012 in 18 Dutch hospitals (about 
20% of all Dutch hospitals). Hospitals were randomly 
selected using a stratified sample based on geographical 
regions and hospital type. Two academic hospitals, four 
teaching hospitals and 12 general hospitals were 
included in this study. All hospitals consented to the 
study and were informed about further practical issues. 
Twelve observers participated in this study. 
Inter-observer variability was not measured, but limited 
by training of observers prior to the start of the observa- 
tions. Moreover, regular feedback meetings were held 
where observers exchanged experiences and discussed 
how to deal with certain situations and observations at 
the OR. A random selection was made from all adult 
patients scheduled for elective surgery on the day of the 
observation. This selection was made by the observers 
who were instructed to attend as many different surger- 
ies as possible while ensuring they were present in the 
OR before the start of each surgery, which was essential 
in order to be able to observe the TOP procedure. The 
goal was to have 10 observation days per hospital at 
intervals of 4-6 weeks, and to observe 6-10 surgical pro- 
cedures per day, preferably involving different surgeons 
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and different procedures. One observer per surgical pro- 
cedure evaluated whether the TOP was carried out 
before anaesthesia, using a standardised recording form 
that covered the various aspects of doing the TOP: 
checking the patient, procedure, and side/site, attention 
of the team (focus), completeness of the team, interrup- 
tions, and several background variables such as the type 
of surgical procedure, the patient's age and sex. The OR 
team was not aware of the exact subject matter of the 
observation; the observer was instructed to introduce 
the study in abstract terms, referring to it as a study 
about the surgical process in general. 

TOP compliance 

The outcome measure was whether the TOP was done 
correcdy and was dichotomous (yes/no). This variable 
was used to examine mean TOP compliance and the 
changes in compliance during the study period. A 
correct TOP consists of three checks: patient, procedure 
and side/site. Since all three checks are equally import- 
ant for preventing wrong surgery, the TOP was only 
deemed correct when all three checks were performed. 
Furthermore, during a TOP the entire OR team gathers 
around the patient and the surgeon asks the patient 
his/her name, the type of procedure and the side/site 
of the procedure. 

Four independent variables were included so that any 
association with compliance could be determined. The 
type of hospital was categorised into academic, teaching, 
and general. In the Netherlands, teaching hospitals 
provide specialised medical care and are committed to 
training and education. The level of care can be charac- 
terised as complex and lies between that of general hos- 
pitals and academic centres. Hospital size was 
operationalised as the number of beds in the hospital 
(a continuous variable). Surgical specialty was added as 
a categorical variable with general surgery as the refer- 
ence category. Focus (yes/no) was included to measure 
the degree to which the OR team was paying full atten- 
tion to the TOP and was not performing any other activ- 
ities during the TOP. In addition, the patient 
characteristics 'age' and 'sex' were included as covari- 
ates. Completeness of the team (yes/no) was added as 
an explorative analysis. The complete team in this study 
was seen as the group of persons that performed the 
surgery on the patient. To be able to perform a TOP 
correcdy, the complete team was present during the 
TOP. When this was not the case, meaning that one or 
more persons joint the team ctfter the TOP had been 
completed, team completeness was scores as 'no'. 

Statistical analyses 

Descriptive analyses were performed to obtain a picture 
of the study population, mean TOP compliance, 
changes in compliance over time, mean compliance for 
the different hospital types, mean compliance for the 
different surgical specialties, and the focus and com- 
pleteness of the team during the TOP. 



A multilevel logistic regression analysis with two levels 
was used to determine whether TOP compliance 
changed between the 10 measuring moments. Multilevel 
analysis was chosen to correct for the fact that the surgi- 
cal procedures are not independent from each other, 
but clustered within hospitals. Time was modelled by 
adding 10 indicator variables for the measurement 
moments (removing the intercept from the model); 
trends were tested using polynomial contrasts (to the 
fourth order) to study changes over time. Variance and 
intraclass correlations (ICCs) were calculated to assess 
the clustering of TOP compliance at the hospital and 
surgical procedure level. An ICC of 20% was seen as 
moderate. '" The changes over time were also analysed 
for the different hospital types to determine the relation- 
ship between hospital type and the changes in TOP 
compliance. Separate logistic multilevel analyses were 
performed for each independent variable to analyse the 
effects of the independent variables 'hospital size' and 
'surgical specialty'; this was necessary because not 
enough units at the highest level (hospitals) were avail- 
able to have more than one independent variable in a 
model.'^" There were not enough units at the highest 
level (hospitals) to model the effect of hospital type on 
the TOP score in the pooled analyses. Age and sex of 
the patient were added as covariates in all analyses. All 
descriptive analyses were performed using SPSS version 
PASW Statistics V.18. The multilevel analyses were per- 
formed using MlwiN V2.24 (using PQL, second order, 
unconstrained level 1 variance, and options) . 

RESULTS 

Descriptive analyses 

A total of 1281 surgical procedures were observed at the 
participating hospitals. After patients younger than 18 
were excluded, 1232 observations remained for analysis. 
Ages ranged from 18 to 96. The gender distribution was 
41.4% male, 53.8% female, and 4.8% not registered. 
The range in types of surgical procedures was broad; 
observers had been instructed to observe different pro- 
cedures and observed surgical procedures of in total 13 
different specialties. Mean compliance with the TOP 
during the total study period was 71.3%. Descriptive ana- 
lyses showed that TOP compliance did not improve 
during the study period. There was a large spread 
between hospitals: one of the hospitals never performed 
the TOP correcdy and two had mean compliance rates 
higher than 90%. A low mean TOP compliance (48%) 
was found at the ninth measuring moment for all the 
participating hospitals. The academic hospitals had a 
mean compliance rate of 42.1%, teaching hospitals 
76.2% and general hospitals 73.9%. Differences between 
specialties were shown to exist: trauma, gastroenterology 
and hepatology and ear, nose and throat medicine 
(ENT) had the highest compliance rates. 
Anaesthesiology, cardiothoracic surgery and cosmetic 
surgery had the lowest compliance rates. In 44% of the 
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observations tiie team was not focused on the TOP and 
in 56%, the team was incomplete. 

Multilevel regression analyses 

In the first multilevel regression analysis, the changes in 
TOP compliance were tested. The effect was statistically 
significant for the fourth-order polynomial (p<0.01), 
meaning that TOP compliance was not linear but fluctu- 
ated over time and no clear trend was observed. 
Furthermore, there were large differences between the 
measuring moments and between individual hospitals 
(see figure 1). The multilevel analysis shows that 44% 
(ICC=44.01) of the total variance in TOP compliance 
can be attributed to the differences between the individ- 
ual hospitals. Adding hospital type to the analysis caused 
tiie ICC to drop to 40.11 (40%; see table 1). 

When correcting for age and sex of the patient, the 
ICC dropped to 26% (ICC=26.58). The relationship 
between the age of the patient and the TOP was found 
to be significant (p<0.05). This relationship was tested 
and found to be linear. Based on the results described 
above, there was no rationale to correct for time (meas- 
urement moments) in further analyses. Observations 
from the different measurement moments were pooled 
in the remaining analyses. 

Separate analyses were performed for the independ- 
ent variables 'hospital size', 'surgical specialty', and 
'focus'. No statistically significant relationship was found 
between hospital size and TOP compliance (data not 
shown in tables). A positive relationship was found 
between patients undergoing ENT surgery and the TOP 
(reference=general surgery; p<0.01). Another positive 
relationship was found between patients undergoing 
ophthalmic surgery and the TOP (reference=general 
surgery; p<0.05; see table 2). This indicates that TOP 
compliance is significantly higher in patients undergo- 
ing ENT surgery or ophthalmic surgery compared with 
patients undergoing general surgery. The relationship 
between the age of the patient and TOP compliance was 
found to be significant (p<0.05) in all analyses. This 
indicates that TOP compliance decreases with the 



patient age. The TOP is performed correctiy less often 
for older patients. An additional analysis was performed 
based on these results to determine which of the three 
individual checks of the TOP attributed most to the 
negative relationship between the age of the patient and 
TOP compliance. Table 3 shows the results of the add- 
itional analysis. The check procedure contributes most 
to the negative relationship between age of the patient 
and TOP compliance, this check is more often skipped 
when an older patient is involved. The relationship 
between the focus of the team during the TOP and the 
correct execution of the TOP is shown in table 4. There 
is a positive significant relationship between focus and 
TOP compliance, which indicates that the TOP is more 
often correctiy executed when the entire team is 
focused on the TOP and not performing any other activ- 
ities at the same time. 



DISCUSSION 

The objective of this study was to investigate the compli- 
ance at Dutch hospitals with the national guidelines of a 
TOP set by the Safety Program and how this changed 
over the final year of the programme. Furthermore, we 
studied variables that might be associated with compli- 
ance. This study found a mean TOP compliance of 
71.3%. There was no linear trend in the TOP compli- 
ance during the study period. Large differences were 
found between and within individual hospitals, which 
were partly influenced by age of the patient. The type of 
hospital was associated with the TOP compliance: aca- 
demic hospitals had lower compliance rates than 
general and teaching hospitals. Given the low number of 
academic hospitals in this study (N=2), these findings 
cannot be generalised to academic hospitals as a whole. 
ENT medicine and ophthalmological surgery had 
higher TOP compliance than the reference group 
(general surgery). No statistically significant relationship 
between TOP compliance and hospital size was found. 
The TOP was correctly performed more often when the 
OR team was focused on it. The negative relationship 



Figure 1 Trend in the time-out 
procedure compliance per 
hospital type, and overall mean 
(n=1232). 
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Trend in the time-out procedure per tiospital type (n=1232; 18 liospitals) 
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Trend overall 



Trend per hospital type 



N 



General hospitals 



Mean % 95% CI 



N Mean % 95% CI 



Teaching hospitals 
N Mean % 95% CI 



Academic hospitals 
N Mean % 95% CI 



Fixed effects 
TOP (constant) 

IVIIVI 1 

IVIIVI 2 

IVIIVI 3 

IVIIVI 4 

MM 5 

MM 6 

MM 7 

MM 8 

MM 9 

MM 10 
Random effects 
Variance components: 

ICC 

Hospital (level 2) 
Surgical procedure (level 1 ) 
*p<0.05. 

Raw data for the remaining variables (specialty, focus and individual checl<s) 
ICC, intraclass correlation coefficient; IVIIVI, measurement moment; TOP, time 
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Table 2 Relationship between surgical specialties (n=1130; 18 hospitals) and compliance with the time-out procedure 



Model 0 (time-out procedure+age+sex) 



Model 1 (model 0+specialties) 



Fixed effects 

Time out procedure (constant) 
Specialties — general surgery 
Specialties — gynecology 
Specialties — ENT 
Specialties — ophthalmology 
Specialties — orthopedic surgery 
Specialties — urology 
Specialties — other 
Patient age 
Patient sex 
Random effects 
Variance components: 

Intraclass correlation 

Hospital (level 2) 

Surgical procedure (level 1) 



Estimate (SE) 
1.173 (0.268) 



-0.01 1 (0.004)* 
0.064 (0.153) 



25.331 

1.116 (0.422)* 
0.996 (0.042) 



Estimate (SE) 
1.196 (0.269) 
Reference 
0.050 (0.264) 
0.905 (0.316)* 
0.616 (0.302)* 
0.163 (0.241) 
0.084 (0.287) 
0.046 (0.279) 
-0.011 (0.004)* 
0.074 (0.155) 



25.499 

1.126 (0.426)* 
1 .006 (0.043) 



*p<0.05. 

ENT, ear, nose and throat medicine. 



between age of the patient and the TOP indicates that 
higher patient age is associated with lower TOP compli- 
ance. Of all the observed TOPs, 44% were performed 
without the focus of the entire team, and the team was 
not complete in 56% of the TOPs. 

A wide range in compliance rates for surgical check- 
lists can be found in previous studies, ranging from 12% 
to 99% with a mean of 75%.'^^"'^'^ The compliance rate 
(71.3%) found in our study is slightly lower than the 
mean rate found in other studies. 

We found a difference in TOP compliance between 
the different types of hospitals. The general and teach- 
ing hospitals hardly differed from each other, which is 
interesting because a previous study'^^ found teaching 
hospitals to be better at implementing checklists than 
general hospitals. According to the organisational learn- 
ing theory, the availability of knowledge in an organisa- 
tion contributes to the adoption of innovations.'^'' 



Teaching hospitals are learning environments, aimed at 
spreading and developing knowledge; better compliance 
can therefore be expected in teaching hospitals. We 
found that academic hospitals showed lower TOP 
compliance. 

The literature is inconsistent about the influence of 
hospital size on the use of checklists. Some argue that 
larger hospitals are better developed and use standar- 
dised processes, which increases the quality of the hos- 
pital more often, whereas others conclude that 
smaller hospitals implement checklists better.^^ We 
found no relationship between TOP compliance and 
hospital size. The high ICC rates found in this study 
suggest that the differences between individual hospitals 
are high, and differences in compliance cannot be 
explained by general hospital characteristics such as hos- 
pital size. The differences between individual hospitals 
need to be examined in further research, but possible 



Table 3 Age effects for the three different checks in the time-out procedure: checking the patient (n=1074), the procedure 
(n=1074), and the side/site (n=1074) 



Model 0 (check 
patient+age+sex) 



Model 1 (check 
procedure+age+sex) 



Model 2 (check 
side/site+age+sex) 



Fixed effects 
Check patient (constant) 
Check procedure (constant) 
Check side/site (constant) 
Patient's age 
Patient's sex 
Random effects 
Variance components: 

Intraclass correlation 

Hospital (level 2) 

Surgical procedure (level 1) 
*p<0.05. 



Estimate (SE) 
3.499 (0.334) 



0.008 (0.008) 
-0.185 (0.288) 



27.172 

1 .228 (0.623)* 
0.834 (0.036) 



Estimate (SE) 

2.276 (0.282) 

-0.021 (0.006)* 
0.124 (0.198) 



24.990 

1 .096 (0.464)* 
0.922 (0.040) 



Estimate (SE) 



2.739 (0.204) 
0.012 (0.007)* 
0.160 (0.246) 



10.854 

0.401 (0.236)* 
0.950 (0.041) 
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TabiG 4 RGlationship bGtwGen focus (n= 


=1074; 18 hospitals) during the time-out procedure and compliance with the time-out 


nrnrpdurp 








Model 0 (time-out procedure+age+sex) 


Model 1 (model 0+focus) 


Fixed effects 


tsTimaie ^ot^ 


tsTimaie v^tj 


Time-out procedure (constant) 


1 .04U ^U.lDoj 


1.4/1 (U.loo; 


Focus 




n (^AT /n 1 71 \* .^^^1 
u.oo/ ^U. 1 / 1 ) ^^^^H 


Patient's age 


— u.uuo yj.uyjoj 


— u.uuo ^U.UUO^ ^^^^^H 


Patient's sex 


-0.012 (0.162) 


-0.016(0.163) 


Random effects 






Variance components: 






Intraclass correlation 


8.971 




Hospital (level 2) 


0.324 (0.154)* 


0.286(0.140)* 


Surgical procedure (level 1) 


0.968 (0.042) 


0.966 (0.042) 





explanations might be found in different organisational 
structures, the creation of awareness among healthcare 
stctff and differences in speaking-up cultures between 
hospitals. 

The relationship found between surgical specialties 
and the TOP is different from the results of previous 
studies. One study showed a difference between sur- 
geons and anesthesiologists""* and another study showed 
no difference between surgical specialties at all.''^^ The 
TOP is a standardised procedure, and the way in which 
it should be carried out does not depend on the surgical 
specialty performing the procedure or the patient 
characteristics. Compliance with the TOP varied 
between different specialties and was lowest among 
general surgery teams. One explanation for these differ- 
ences could be that not all medical disciplines and their 
scientific communities have placed the same amount of 
weight on a thorough implementation of the Sctfety 
Program. If so, this could have had an influence on the 
sense of urgency experienced by different specialties to 
comply with TOP in their daily functioning. Further 
research that includes specialty-specific factors is needed 
to verify and deepen our findings. The negative relation- 
ship between TOP compliance and the age of the 
patient was an unexpected result, since the TOP should 
be executed in the same way for all patients. In particu- 
lar, the exact surgical procedure that would be carried 
out was less often verified with elderly patients. 
Explanations might be found in factors inherentiy asso- 
ciated with the elderly patient themselves. For example, 
elderly patients might be less able to verbally express 
themselves to healthcare staff. On the other hand, expla- 
nations might be found in factors that are associated 
with the medical procedure itself. For example, the level 
of standardisation of procedures that are commonly per- 
formed in the elderly population (such as 
hip-replacement surgery or cataract surgery) is relatively 
high and it is unclear what effect this has on compliance 
with TOP. Elderly people are a vulnerable group with a 
higher risk of complications aifter surgery, therefore 
further in-depth research is important to explain the dif- 
ferences in compliance for different age groups. 



Completeness and focus are important factors in the 
TOP and performing it when team members are busy 
with other activities creates a risk. Our study showed that 
focus in the team contributes to the TOP being per- 
formed correctiy. However, there was poor focus on the 
TOP in almost half of the surgical procedures observed. 
Several possible causes could be underlying to poor 
focus during the TOP, which was observed frequentiy in 
our study. First, there could have been a lack of aware- 
ness of the importance of the TOP among healthcare 
staff. Regular emphasis on the importance of the TOP 
during team meetings or during the joint briefing at the 
start of a new working day could help raise awareness. 
Second, when surgery schedules are tight, healthcare 
staff might experience time pressure. In trying to keep 
up with the schedule and being efficient, healthcare 
staff might be tempted to perform multiple tasks simul- 
taneously which in turn could negatively affect compli- 
ance with TOP. 

On the basis of these results, it seems that hospitals 
still have a lot to gain by carrying out the TOP properly. 
Qualitative research methods could provide insight into 
the underlying reasons and incentives of why healthcare 
staff perform the TOP in the way they currentiy do. This 
type of research could complement and deepen the 
findings that were presented in the current study. 

Strengths and limitations 

Our study was the first to evaluate TOP compliance over 
time through observations in the OR and look into the 
factors associated with compliance. Our dependent vari- 
able was a process indicator, because the incidence of 
wrong surgery is too low to be observed with our study 
design. Based on the literature, it seems fair to assume 
that higher TOP compliance can contribute to a 
decrease in the incidence of wrong surgery,^ ' although 
this study gives no information about the actual number 
of wrong surgeries and TOP compliance might not be 
the only factor in the reduction of wrong surgery. 

This study has several limitations. First, the presence 
of the observer might have influenced the behaviour of 
the OR staff and indirectiy our dependent variable TOP. 
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However, the design of our study aimed to prevent this 
potential observer bias, because the precise goal of the 
observations was not known to the OR team. Second, a 
potential selection bias can be found in the selection of 
surgical procedures on the observation days. Surgical 
procedures were selected based on practical considera- 
tions: the day of the week, the duration of the procedure 
and the OR schedule. The relationships found between 
different specialisms might be partially overestimated, 
because the same surgical teams were sometimes 
observed on the same day or on different observation 
days. However, the overall goal was to observe as may dif- 
ferent surgical procedures with different teams as pos- 
sible, in order to limit potential selection bias. Third, 
there is no information available about the changes in 
compliance during the first period of the Sctfety 
Program, and hospitals may have made progress during 
this period. 

CONCLUSIONS 

The mean TOP compliance was 71.3% during the final 
year of the Safety Program and no improvement in com- 
pliance over time was found. Large differences were 
found between hospitals, and these differences were 
influenced by age of the patient. Compliance was influ- 
enced by several factors: hospital type, surgical specialty, 
age of the patient and focus of the team during the 
TOP. Furthermore, in almost half the TOPs, the team 
was not focused on the TOP or the team was incom- 
plete. Despite the fact that almost three quarter of 
operations are preceded by a TOP, hospitals need to 
make an effort to improve TOP compliance and the way 
in which the TOP is carried out in order to prevent 
wrong surgery from happening in the future. 
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